Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters
نویسنده
چکیده
One of the attractions of neural network approaches to pattern recognition is the use of a discrimination-based training method. We show that once we have modified the output layer of a multilayer perceptron to provide mathematically correct probability distributions, and replaced the usual squared error criterion with a probability-based score, the result is equivalent to Maximum Mutual Information training, which has been used successfully to improve the performance of hidden Markov models for speech recognition. If the network is specially constructed to perform the recognition computations of a given kind of stochastic model based classifier then we obtain a method for discrimination-based training of the parameters of the models. Examples include an HMM-based word discriminator, which we call an 'Alphanet' .
منابع مشابه
On classification improvement by using an approximate discriminative hidden Markov model Mejoramiento de la clasificación usando un modelo oculto de Markov discriminativo aproximado
HMMs are statistical models used in a very successful and effective form in speech recognition. However, HMM is a general model to describe the dynamic of stochastic processes; therefore it can be applied to a huge variety of biomedical signals. Usually, the HMM parameters are estimated by means of MLE (Maximum Likelihood Estimation) criterion. Nevertheless, MLE has as disadvantage that the dis...
متن کاملLattice segmentation and minimum Bayes risk discriminative training for large vocabulary continuous speech recognition
Lattice segmentation techniques developed for Minimum Bayes Risk decoding in large vocabulary speech recognition tasks are used to compute the statistics needed for discriminative training algorithms that estimate HMM parameters so as to reduce the overall risk over the training data. New estimation procedures are developed and evaluated for both small and large vocabulary recognition tasks, an...
متن کاملModelling Climatic Parameters Affecting the Annual Yield of Rheum Ribes Rangeland Species using Data Mining Algorithms
Identification of climatic characteristics affecting the annual yield of Rheum Ribes can be useful in management and development of this species in the rangelands. In this research, the annual yield of this species in Khorasan-Razavi province based on 74 climatic parameters during a ten-year period evaluated and affecting climatic parameters extracted using data mining methods. First, the role ...
متن کاملApproximations to the MMI criterion and their effect on lattice-based MMI
Although maximum mutual information (MMI) training has been used for hidden Markov model (HMM) parameter estimation for more than twenty years ([2], [8], [5], [9], and [14]), it has recently become an essential part of the acoustic modeling repertoire thanks to the refinements introduced by Woodland and Povey ([16] and [11]). The earliest incarnations of MMI worked well on small vocabulary task...
متن کاملSELECTIVE TRAINING FOR HIDDEN MARKOVMODELS with APPLICATIONS to SPEECHCLASSIFICATIONbyLevent
Traditional maximum likelihood estimation of hidden Markov model parameters aims at maximizing the overall probability across the training tokens of a given speech unit. Therefore, it disregards any interaction and biases across the models in the training procedure. Often the resulting model parameters do not result in minimum error classiication in the training set. A new selective training me...
متن کامل